1. Field of the Invention
The present invention relates to an artificial neural network structure. More specifically, the present invention relates to an interconnecting neural network system, an interconnecting neural network structure construction method, a self-organizing neural network structure construction method having a novel network form excellent in flexibility of structure and in facility of training, and construction programs therefor.
2. Related Art
As a conventional artificial neural network structure, an artificial neural network structure having a fixed network form such as a layered network that inputs a single input vector and adjusting network parameters such as weight vectors is normally known. As a method of adjusting the network parameters, a back-propagation method for iteratively updating network parameters is widely used (D. E. Rumelhart, G. E. Hinton, and R. J. Williams, “Learning internal representations by error propagation,” In D. E. Rumelhart and J. L. McClelland (Eds.), “Parallel Distributed Processing: explorations in the microstructure of cognition,” 1. Chapter 8, Cambridge, Mass. MIT Press, 1986).
However, for the conventional artificial neural network structure, an iterative training scheme for iteratively updating network parameters, such as the back-propagation method, is employed as the method of adjusting network parameters. Due to this, the conventional artificial neural network structure has the following disadvantages: (1) it takes considerable time to update network parameters before the connection between input and output is established; (2) a solution obtained as a result of updating network parameters tends to be a local minimum and it is difficult to obtain a correct solution; and (3) it is difficult to realize a robust additional training method.
Furthermore, the conventional artificial neural network structure has disadvantages in that the structure is inferior in network configuration flexibility and no practical, effective method capable of handling a plurality of input vectors is established yet.
As a conventional method of handling a plurality of input vectors in the artificial neural network structure, a modular approach for modularizing various neural networks (or agents) and integrating the neural network (or agent) modules is proposed (S. Haykin, “Neural Networks: A Comprehensive Foundation,” Macmillan College Publishing Co. Inc., N.Y., 1994).
Even if such an approach is used, an artificial neural network structure having a fixed network form based on an iterative training scheme is used for every network module similarly to the conventional artificial neural network structure. Therefore, the approach is faced with the substantial disadvantages stated above, as well.